Systems and Methods for Updating an Electronic Map

ABSTRACT

Systems and methods for updating an electronic map of a facility are disclosed. The electronic map includes a set of map nodes. Each map node has a stored image data associated with a position within the facility. The method includes collecting image data at a current position of a self-driving material-transport vehicle; searching the electronic map for at least one of a map node associated with the current position and one or more neighboring map nodes within a neighbor threshold to the current position; comparing the collected image data with the stored image data of the at least one of the map node and the one or more neighboring map nodes to determine a dissimilarity level. The electronic map may be updated based at least on the collected image data and the dissimilarity level. The image data represents one or more features observable from the current position.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.16/185,077, filed Nov. 9, 2018, which claims the benefit of U.S.Provisional Patent Application No. 62/584,238, filed Nov. 10, 2017, andtitled “SYSTEMS AND METHODS FOR UPDATING AN ELECTRONIC MAP”. The contentof each of U.S. patent application Ser. No. 16/185,077 and U.S.Provisional Patent Application No. 62/584,238 is incorporated herein byreference for all purposes.

TECHNICAL FIELD

The described embodiments relate to simultaneous localization andmapping, and in particular, to systems and methods for updating anelectronic map.

BACKGROUND

The following paragraphs are not an admission that anything discussed inthem is prior art or part of the knowledge of persons skilled in theart.

An automated guided vehicle (“AGV”) operates along pre-determined paths,defined by specific infrastructure, such as a magnetic strip. AGVscannot deviate from the pre-determined path. In contrast, a self-drivingmaterial-transport vehicle can operate without specific infrastructureand are not limited to a pre-determined path. A self-drivingmaterial-transport vehicle can sense its environment and navigate itsenvironment without human input.

A self-driving material-transport vehicle typically stores an electronicmap of its environment and identifies its location within theenvironment based the stored electronic map and what it has sensed inits environment. However, the electronic map can become out of date asthe environment changes.

SUMMARY

The various embodiments described herein generally relate to methods(and associated systems configured to implement the methods) forupdating an electronic map of a facility. The method includes collectingimage data at a current position of a self-driving material-transportvehicle; searching the electronic map for at least one of (i) a map nodeassociated with the current position and (ii) one or more neighboringmap nodes within a neighbor threshold to the current position, comparingthe collected image data with the stored image data of the at least oneof the map node and the one or more neighboring map nodes to determine adissimilarity level; and in response to determining the dissimilaritylevel exceeds a dissimilarity threshold, updating the electronic mapbased at least on the collected image data. The image data representsone or more features observable from the current position. Theelectronic map includes a set of map nodes and each map node includes astored image data associated with a position within the facility.

In some aspects, comparing the collected image data with the storedimage data to determine the dissimilarity level includes comparing thecollected image data with the stored image data of the map node todetermine the dissimilarity level.

In some aspects, searching the electronic map includes, in response todetermining the electronic map does not include the map node associatedwith the current position, searching the electronic map for the one ormore neighboring map nodes within the neighbor threshold to the currentposition.

In some aspects, comparing the collected image data with the storedimage data includes comparing the collected image data with the storedimage data of the one or more neighboring map nodes to determine thedissimilarity level.

In some aspects, comparing the collected image data with the storedimage data includes: determining a number of mismatched points betweenthe collected image data and the stored image data; and determining thedissimilarity level based on the number of mismatched points and a totalnumber of points.

The mismatched points may include one or more points associated with oneof a projection error and a reprojection error between the collectedimage data and the stored image data.

The reprojection error may include a mismatch between the collectedimage data and the stored image data exceeding an offset threshold.

The electronic map may include the map node associated with the currentposition. In some aspects, updating the electronic map based at least onthe collected image data includes replacing the stored image data of themap node with the collected image data.

In some aspects, updating the electronic map based at least on thecollected image data includes in response to determining the electronicmap does not include the map node associated with the current position,adding a new map node to the electronic map for the current position andstoring at the new map node the collected image data.

In some aspects, adding the new map node includes connecting the new mapnode to at least two other map nodes of the electronic map.

In some aspects, the method further includes determining that theelectronic map includes one or more excessive neighboring map nodesrelative to the new map node, and in response to determining theelectronic map includes the one or more excessive neighboring map nodes,removing the one or more excessive neighboring map nodes from theelectronic map. The one or more excessive neighboring map nodes can be amap node associated with a position that is less than an excessiveneighbor distance from the current position of the new map node.

The excessive neighbor distance may include a minimum distance betweentwo map nodes of the electronic map.

In some aspects, updating the electronic map based at least on thecollected image data includes, for each neighboring map node of the oneor more neighboring map nodes, comparing the collected image data withthe stored image data to determine the dissimilarity level; and inresponse to determining the dissimilarity level exceeds a node removalthreshold, removing that neighboring map node from the electronic map.

The node removal threshold may be greater than the dissimilaritythreshold.

The neighbor threshold may include a maximum distance between a positionof two map nodes. The neighbor threshold may include a maximum number ofmap nodes.

The dissimilarity threshold may include a minimum mismatch between thecollected image data and the stored image data to indicate theelectronic map is outdated.

In some aspects, the method further includes determining thedissimilarity threshold based on at least one of a facility type of thefacility and one or more prior updates to the electronic map.

In some aspects, collecting the image data at the current position ofthe self-driving material-transport vehicle includes operating theself-driving material-transport vehicle to collect the image data whilethe self-driving material-transport vehicle is conducting a mission.

The position of each map node can include a coordinate and a headingwithin a coordinate system assigned to the electronic map.

In another aspect, a system for updating an electronic map of a facilityis described. The electronic map includes a set of map nodes and eachmap node includes a stored image data associated with a position withinthe facility. The system includes one or more self-drivingmaterial-transport vehicles operable to navigate the facility.

The self-driving material-transport vehicle of the one or moreself-driving material-transport vehicles may include: a memory to storethe electronic map, one or more sensors to collect image data at acurrent position of the at least one self-driving material-transportvehicle, and a processor. The processor is operable to: search theelectronic map for at least one of (i) a map node associated with thecurrent position and (ii) one or more neighboring map nodes within aneighbor threshold to the current position, compare the collected imagedata with the stored image data of the at least one of the map node andthe one or more neighboring map nodes to determine a dissimilaritylevel; and in response to determining the dissimilarity level exceeds adissimilarity threshold stored in the memory, update the electronic mapbased at least on the collected image data. The image data representsone or more features observable form the current positon. The neighborthreshold can be stored in memory.

BRIEF DESCRIPTION OF THE DRAWINGS

Several embodiments will now be described in detail with reference tothe drawings, in which:

FIG. 1 is a diagram of a system of one or more self-drivingmaterial-transport vehicles, according to at least one embodiment;

FIG. 2 is a block diagram of a self-driving material-transport vehicle,according to at least one embodiment;

FIG. 3 is another block diagram of a self-driving material-transportvehicle, according to at least one embodiment;

FIG. 4 is a diagram of an example layout of a facility and self-drivingmaterial-transport vehicles located therein, according to at least oneembodiment;

FIG. 5 is an illustration of an electronic map, according to at leastone embodiment;

FIG. 6 is a method of updating an electronic map of a facility,according to at least one embodiment;

FIG. 7A is a diagram of an example of a laser scan;

FIG. 7B is a diagram of projection error when a laser scan is comparedwith an electronic map, according to at least one embodiment;

FIG. 7C is a diagram of reprojection error when a laser scan is comparedwith an electronic map, according to at least one embodiment;

FIG. 8A is a diagram of a first laser scan;

FIG. 8B is a diagram of a second laser scan;

FIG. 8C is a diagram of an alignment of the first laser scan of FIG. 8Aand the second laser scan of FIG. 8B;

FIG. 9A is a diagram of a self-driving material-transport vehiclelocated in the facility shown in FIG. 5 having a second layout,according to at least one embodiment;

FIG. 9B is a diagram of a self-driving material-transport vehicle in asecond position in the facility having the layout shown in FIG. 9A,according to at least one embodiment;

FIG. 10A is an illustration of replacing stored image data of a map nodeof an electronic map, according to at least one embodiment;

FIG. 10B is an illustration of a updating neighbor map nodes of anelectronic map, according to at least one embodiment;

FIG. 11A is an illustration of adding a map node to an electronic map,according to at least one embodiment;

FIG. 11B is an illustration of removing an excessive neighboring mapnode from an electronic map, according to at least one embodiment; and

FIG. 12 is a flow diagram depicting a method for zone-based remapping ofa facility, according to at least one embodiment.

The drawings, described below, are provided for purposes ofillustration, and not of limitation, of the aspects and features ofvarious examples of embodiments described herein. For simplicity andclarity of illustration, elements shown in the drawings have notnecessarily been drawn to scale. The dimensions of some of the elementsmay be exaggerated relative to other elements for clarity. It will beappreciated that for simplicity and clarity of illustration, whereconsidered appropriate, reference numerals may be repeated among thedrawings to indicate corresponding or analogous elements or steps.

DETAILED DESCRIPTION

Self-driving material-transport vehicles identify their location withinthe environment based on a stored electronic map of its environment andwhat it has sensed in its environment. The process of localizationinvolves taking a scan of the environment and then comparing knownfeatures of the environment found in the scan with known features of theenvironment in the electronic map in order to identify the position ofthe self-driving material-transport vehicle. However, features found inthe scan can be different from known features in the electronic map ifthe environment has changed. The environment may change when objects aremoved.

Since self-driving material-transport vehicles are driving around thefacility to execute missions, they can simultaneously observe theenvironment and update the electronic map based on their observations.Self-driving material-transport vehicles can be operated to update theelectronic map during missions or between missions. As a result, theelectronic map can be continuously updated.

However, the electronic map can grow in size with continuous updates,eventually becoming unmanageable. In order to maintain a proper map, itis desirable to represent features of the environment in an electronicmap efficiently.

Referring to FIG. 1 , there is shown a system 100 of one or moreself-driving material-transport vehicles 110, according to at least oneembodiment. The system 100 can include one or more self-drivingmaterial-transport vehicles 110, a fleet management system 120, anetwork 130, and a system storage component 140. While FIG. 1 shows thesystem 100 having two self-driving material-transport vehicles 110 a and110 b for illustrative purposes. The system 100 can include one or moreself-driving material-transport vehicles 110.

According to some embodiments, a fleet management system 120 may be usedto provide a mission to a self-driving material-transport vehicle 110.The fleet management system 120 has a processor, memory, and acommunication interface (not shown) for communicating with the network130. The fleet management system 120 uses the memory to store computerprograms that are executable by the processor (e.g. using the memory) sothat the fleet management system 120 can communicate information withother systems, and communicate with one or more self-drivingmaterial-transport vehicles 110. In some embodiments, the fleetmanagement system 120 can also generate missions for the self-drivingmaterial-transport vehicles 110.

Any or all of the self-driving material-transport vehicles 110 and thefleet management system 120 may communicate with the network 130 usingknown telecommunications protocols and methods. For example, eachself-driving material-transport vehicle 110 and the fleet managementsystem 120 may be equipped with a wireless communication interface toenable wireless communications according to a WiFi protocol (e.g. IEEE802.11 protocol or similar).

According to some embodiments, the system storage component 140 canstore information about the self-driving material-transport vehicles 110as well as electronic maps of the facilities within which theself-driving material-transport vehicles 110 operate. Electronic mapscan be stored in the system storage component for subsequent retrievalby individual self-driving material-transport vehicles 110. Individualself-driving material-transport vehicles 110 can download electronicmaps from the system storage component 140 via network 130.

Electronic maps can be human generated. For example, a CAD file can beimported and form the basis for an electronic map. In another example,an electronic map can be built by driving the self-drivingmaterial-transport vehicle 110 around the facility for the purpose ofbuilding an electronic map.

Referring to FIG. 2 , there is shown a block diagram of a self-drivingmaterial-transport vehicle 110, according to at least one embodiment.The self-driving material-transport vehicle 110 generally includes acontrol system 210 and a drive system 230.

The control system 210 can include a processor 212, memory 214,communication interface 216, and one or more sensors 220. The controlsystem 210 enables the self-driving material-transport vehicle 110 tooperate automatically and/or autonomously. The control system 210 canstore an electronic map that represents the environment of theself-driving material-transport vehicle 110, such as a facility, in thememory 214.

According to some embodiments, the communication interface 216 can be awireless transceiver for communicating with a wireless communicationsnetwork (e.g. using an IEEE 802.11 protocol or similar).

One or more sensors 220 may be included in the self-drivingmaterial-transport vehicle 110 to obtain data about the environment ofthe self-driving material-transport vehicle 110. For example, accordingto some embodiments, the sensor 220 may be a LiDAR device (or otheroptical/laser, sonar, radar range-finding such as time-of-flightsensors). The sensor 220 may be optical sensors, such as video camerasand systems (e.g., stereo vision).

According to some embodiments, the self-driving material-transportvehicle 110 may receive a mission from a fleet management system 120 orother external computer system in communication with the self-drivingmaterial-transport vehicle 110 (e.g. in communication via thecommunication interface 216). In this case, the mission contains one ormore waypoints or destination locations. Based on the waypoint ordestination location contained in the mission, the self-drivingmaterial-transport vehicle 110, based on the control system 210, canautonomously navigate to the waypoint or destination location withoutreceiving any other instructions from an external system. For example,the control system 210, along with the sensors 220, enable theself-driving material-transport vehicle 110 to navigate without anyadditional navigational aids such as navigational targets, magneticstrips, or paint/tape traces installed in the environment in order toguide the self-driving material-transport vehicle 110.

For example, the control system 210 may plan a path for the self-drivingmaterial-transport vehicle 110 based on a destination location and thelocation of the self-driving material-transport vehicle 110. Based onthe planned path, the control system 210 may control the drive system230 to direct the self-driving material-transport vehicle 110 along theplanned path. As the self-driving material-transport vehicle 110 isdriven along the planned path, the sensors 220 may update the controlsystem 210 with new images of the environment of the self-drivingmaterial-transport vehicle 100, thereby tracking the progress of theself-driving material-transport vehicle 110 along the planned path andupdating the location of the self-driving material-transport vehicle110.

Since the control system 210 receives updated images of the environmentof the self-driving material-transport vehicle 110, and since thecontrol system 210 is able to autonomously plan the self-drivingmaterial-transport vehicle's path and control the drive system 230, thecontrol system 210 is able to determine when there is an obstacle in theself-driving material-transport vehicle's path, plan a new path aroundthe obstacle, and then drive the self-driving material-transport vehicle110 around the obstacle according to the new path.

The positions of the components 210, 212, 214, 216, 220, and 230 of theself-driving material-transport vehicle 110 are shown for illustrativepurposes and are not limited to the positions shown. Otherconfigurations of the components 2 f 10, 212, 214, 216, 220, and 230 arepossible.

Referring to FIG. 3 , there is shown a block diagram of a self-drivingmaterial-transport vehicle 110, according to at least one embodiment.The drive system 230 includes a motor and/or brakes connected to drivewheels 232 a and 232 b for driving the self-driving material-transportvehicle 110. According to some embodiments, the motor may be an electricmotor, a combustion engine, or a combination/hybrid thereof. Accordingto some embodiments, there may be one motor per drive wheel, forexample, one for drive wheel 232 a and one for drive wheel 232 b.Depending on the particular embodiment, the drive system 230 may alsoinclude control interfaces that can be used for controlling the drivesystem 230. For example, the drive system 230 may be controlled to drivethe drive wheel 232 a at a different speed than the drive wheel 232 b inorder to turn the self-driving material-transport vehicle 110. Differentembodiments may use different numbers of drive wheels, such as two,three, four, etc.

According to some embodiments, additional wheels 234 may be included (asshown in FIG. 3 , the wheels 234 a, 234 b, 234 c, and 234 d may becollectively referred to as the wheels 234). Any or all of theadditional wheels 234 may be wheels that are capable of allowing theself-driving material-transport vehicle 110 to turn, such as castors,omni-directional wheels, and mecanum wheels.

According to some embodiments, the sensors 220 (as shown in FIG. 3 , thesensors 220 a, 220 b, and 220 c may be collectively referred to as thesensors 220) may be optical sensors arranged in a manner to providethree-dimensional (e.g. binocular or RGB-D) imaging.

The positions of the components 210, 212, 214, 216, 220, 230, 232, and234 of the self-driving material-transport vehicle 110 are shown forillustrative purposes and are not limited to the shown positions. Otherconfigurations of the components 210, 212, 214, 216, 220, 230, 232, and234 are possible.

Referring now to FIG. 4 , there is shown a diagram of an example layout400 of a facility. A facility can be any place with a particularpurpose. Example facilities include, but are not limited to,manufacturing plants, warehouses, offices, hospitals, hotels, andrestaurants.

A self-driving material-transport vehicle 110 may travel between variouspoints within a facility to pick-up and deliver items, to monitor thefacility, or to collect data within the facility. For example, in amanufacturing plant, a self-driving material-transport vehicle 110 maypick-up and deliver items to and from the assembly line, stagingstations, and/or storage shelves.

As shown in layout 400, the facility can have a number of obstacles inthe self-driving material-transport vehicle's path, such as shelf 410,staging stations 420 a, 420 b, and 420 c, and pallets 430 a, 430 b and430 c. Obstacles can be permanent objects, such as wall or pillars.Obstacles can also be temporary or transient objects, such as carts,crates, bins, and pallets. The self-driving material-transport vehicle110 may need to navigate around the obstacles in order to complete itsmission.

Self-driving material-transport vehicles 110 a and 110 b each have aposition 402 a and 402 b respectively, within the facility. A position402 a and 402 b can be characterized by a coordinate and a heading inthe coordinate system of the electronic map. For example, if anelectronic map uses an x-y coordinate system, the coordinates can bedefined in terms of x and y and a heading can be yaw (angle) in the x-yplane. As shown in FIG. 4 , positions 402 a and 402 b have a sameheading but different coordinates.

Referring now to FIG. 5 , there is shown an illustration of anelectronic map 500, according to at least one embodiment. An electronicmap 500 can include a set of map vertices such as map nodes 510, 520,530, and 540 and map features 550.

Each map node 510, 520, 530, and 540 can include a stored image dataassociated with a position within the facility. The position stored in amap node can relate to the position of the self-drivingmaterial-transport vehicle 110 at the time that the associated imagedata was observed. According to some embodiments, image data can includean image itself, or a frame of video. According to some embodiments,image data can include LiDAR data. In some embodiments, image data canrelate to data obtained from processing the image, such as featuredetection, extraction and/or classification. Each map node may differ inthe properties and/or source of the stored image data.

Similar to a map node, a map feature 550 is also associated with aposition within the facility. However, map features store user-generatedmap data such as waypoints, zones, and docks, etc.

Referring back to FIG. 4 , map node 520 can be associated with theposition 402 a of self-driving material-transport vehicle 110 a in afacility having layout 400. The stored image data of map node 520 inelectronic map 500 can relate to the presence of obstacle 420 b. Inelectronic map 500, map node 520 is connected along edge 512 to neighbormap node 510. Map node 520 is also connected along edge 522 to neighbormap node 530.

Each vertex can be connected to at least one other vertex, therebyforming a graph. The connection between two vertices can be referred toas an edge 512 and 522. In some embodiments, each vertex is connected toat least two other vertices.

The electronic map 500 is shown for illustrative purposes and is notlimited to the structure shown. Other structures of electronic maps arepossible.

Referring now to FIG. 6 , there is shown a method 600 of updating anelectronic map 600 of a facility, according to at least one embodiment.The method 600 can be carried out by a self-driving material-transportvehicle 110, such as using the processor 212, memory 214, and one ormore sensors 220.

The method 600 begins at step 610, when image data at a current positionof the self-driving material-transport vehicle 110 is collected. Theself-driving material-transport vehicle 110 can collect image data whileconducting a mission in the facility, or while waiting to receive amission. According to some embodiments, the vehicle can execute aremapping mission specifically in order to update the map with thefacility, or within a particular zone of the facility. In someembodiments, the self-driving material-transport vehicle 110 can operatein the facility for the purpose of maintaining the electronic map.

At step 612, image data pertaining to dynamic objects may be detectedand filtered according to some embodiments. Dynamic objects, asdescribed here, are objects within the image data that do not representmap objects. For example, a human pedestrians and other vehicles aregenerally considered to be dynamic objects. According to someembodiments, dynamic objects may be detected based on matching imagedata to known data representing dynamic objects, the temporalcharacteristics of the data (e.g. whether the object is moving betweenone image and another, or within a video; the speed of movement, etc.).Once a dynamic object has been detected in the image data, the dynamicobject may be filtered out so that it is not used within the map.

At step 614, an electronic map 500 is searched for at least one of (a) amap node associated with the current position and (b) one or moreneighboring map nodes within a neighbor threshold to the currentposition. A neighboring map node is a map node that does not share thesame current position but has a position that is relatively nearby orlocal to the current position. In some embodiments, determining whethera map node is a neighbor can be based on a neighbor threshold. In someembodiments, map nodes must be connected to a map node in order to be aneighbor map nodes. In some embodiments, neighbor map nodes do not needto be connected to map nodes in order to be a neighbor map node.

In some embodiments, a neighbor threshold can relate to a distancebetween map node positions. In some embodiments, a neighbor thresholdcan relate to a pre-defined distance between map node coordinates. Forexample, a neighbor threshold can be 15 meters. Two map nodes that havecoordinates that are 20 meters apart would not be considered neighbormap nodes. However, two map nodes that have coordinates that are 10meters apart could be considered neighbor map nodes. In someembodiments, a neighbor threshold in the range of about 10 meters to 20meters is preferred.

In some embodiments, a neighbor threshold can relate to a pre-definedmaximum number of map nodes. For example, a neighbor threshold can be 15meters and a maximum of three map nodes. If a first map node has adistance of 6, 7, 11, 12, and 14 meters away from five other map nodesrespectively, only three map nodes that are shortest in distance fromthe first map node will be considered a neighbor map node. In someembodiments, a neighbor threshold of about 4 map nodes is preferred.

In some embodiments, the electronic map can be searched for a map nodeassociated with the current position. If it is determined that a mapnode associated with the current position does not exist in theelectronic map, the electronic map can be searched again for one or moreneighboring map nodes within the neighbor threshold to the currentposition. That is, in some embodiments, the method 600 may not searchfor one or more neighboring map nodes within the neighbor threshold tothe current position because a map node associated with the currentposition is already found.

At step 616, the image data collected at step 610 is compared with thestored image data. When a map node for the current position exists,comparing the collected image data with the stored image data caninvolve comparing the collected image data with the stored image data ofthe map node for the current position. When a map node for the currentposition does not exist, comparing the collected image data with thestored image data can involve comparing the collected image data withthe stored image data of the one or more neighboring map nodes.

Collected image data can be compared with stored image data (of eitherthe map node or the one or more neighboring map nodes) in order todetermine a dissimilarity level. In some embodiments, comparing thecollected image data with the stored image data can involve determininga number of mismatched points between the collected image data and thestored image data and determining the dissimilarity level based on thenumber of mismatched points and a total number of points of thecollected image data. In some embodiments, the dissimilarity level canbe a ratio of the number of mismatched points to the total number ofpoints of the collected image data.

Mismatched points between the collected image data and the stored imagedata can be a result of projection errors or reprojection errors inlaser scans. Referring now to FIG. 7A, there is shown a diagram of anexample of a laser scan 700. Laser scanner 710 can emit laser beams 732towards obstacles 720 and 730 (as shown in FIG. 7A, the obstacles 720 a,720 b, 730 a, and 730 b may be collectively referred to as 720 and 730).The laser beams 732 reflect back to the laser scanner 710, where thereturn time is measured. The time difference between the initial laserbeam 732 and the return of the reflected laser beam can be used todetermine the distance of obstacles relative to the laser scanner 710.

Referring now to FIG. 7B, there is shown a diagram 700′ of projectionerror when collected image data from a laser scan 700′ is compared withstored image data in an electronic map, according to at least oneembodiment. Projection error can occur when stored image data relates toa laser scan 700′ taken by laser beams 732 a, 732 b, and 732 c in theabsence of obstacle 730 a and the collected image data relates to alaser scan 700′ taken by laser beams 734 a, 734 b, and 734 c in thepresence of obstacle 730 a. The presence or absence of obstacle 730 canresult in a slight offset in the features detected. The slight offset inthe features detected can result in mismatched points. If the mismatchexceeds an offset threshold, the difference is considered a projectionerror.

Referring now to FIG. 7C, there is shown a diagram 700″ of reprojectionerror when collected image data from a laser scan 700″ is compared withstored image data in an electronic map, according to at least oneembodiment. Projection error can occur when stored image data relates toa laser scan 700″ taken by laser beams 732 d, 732 e, and 732 f in thepresence of obstacle 730 b and the collected image data relates to alaser scan 700″ taken by laser beams 736 a, 736 b, and 736 c in theabsence of obstacle 730 b. The presence or absence of obstacle 730 canresult in a slight offset in the features detected. The slight offset inthe features detected can result in mismatched points. If the mismatchexceeds an offset threshold, the difference is considered a reprojectionerror.

In order to compare collected image data and stored image data, it canbe necessary to align a reference from the collected image data with areference frame for the stored image data. Referring to FIGS. 8A, 8B,and 8C, shown therein is a diagram of a first laser scan 810; a diagramof a second laser scan 820; and a diagram of an alignment 830 of thefirst laser scan of FIG. 8A and the second laser scan 820 of FIG. 8B. Insome embodiments, the alignment 830 can be obtained by determining atransformation between the first laser scan 810 and the second laserscan 820. Various techniques for determining a transformation can beused, including but not limited to iterative closest point.

Referring back to FIG. 6 , at step 618, if the dissimilarity levelexceeds a dissimilarity threshold, the electronic map can be updatedbased at least on the image data collected at step 610.

In some embodiments, the dissimilarity threshold relates to a minimummismatch between the collected image data and the stored image data thatindicates the electronic map is outdated. In some embodiments, thedissimilarity threshold can be static, that is, a pre-defineddissimilarity threshold. In some embodiments, the dissimilarly thresholdcan be in the range of approximately 30% to 40% dissimilarity.

In some embodiments, the dissimilarity threshold can be dynamic. Forexample, the dissimilarity threshold can be selected based on a facilitytype of the facility and/or one or more prior updates to the electronicmap. For example, in some embodiments, the dissimilarity threshold canbe higher immediately after an update to the electronic map andconversely, the dissimilarity threshold can be lower if an extendedperiod of time has elapsed since an update to the electronic map. Insome embodiments, a facility that has continuous activity and morefrequent changes can have a higher dissimilarity threshold than afacility that has less activity and relatively less frequent changes.

When the electronic map includes a map node associated with the currentposition, updating the electronic map based at least on the collectedimage data involves replacing the stored image data of the map node withthe collected image data.

Referring now to FIGS. 9A and 10A, there is shown another layout 900 ofthe facility and an updated electronic map 1000. Layout 900 is similarto layout 400 with the exception that obstacle 420 b is removed(indicated by stippled lines) in layout 900. If the self-drivingmaterial-transport vehicle 110 a collects image data of layout 900 atposition 402 a, the electronic map 500 can be updated by replacing thestored image data of map node 520, with the collected image data. Anupdate to the stored image data of map node 520 is indicated by shadingof map node 520 in electronic map 1000 of FIG. 9A.

In some embodiments, collected image data for a map node can be similarto stored image data of a neighboring map node. If stored image data fora map node is replaced with collected image data that is similar tostored image data of a neighboring map node, the electronic map islarger than necessary. In order to reduce redundancy in the electronicmap, the collected image data can be compared with the stored image dataof neighboring map nodes to determine the dissimilarity level. If thedissimilarity level exceeds a node removal threshold, the neighboringmap node can be removed from the electronic map.

Referring now to FIG. 10B, there is shown another updated electronic map1000′. After map node 520 of electronic map 1000 is updated withcollected image data, the collected image data can be compared with thestored image data of map nodes 510 and 530. A dissimilarity levelbetween the collected image data for map node 520 and each of storedimage data for neighboring map nodes 510 and 530 can be determined. Ifthe dissimilarity level with respect to neighboring map node 510 exceedsa node removal threshold, the neighboring map node can be removed. Toremove a map node, edges connecting the map node and the map node itselfcan be removed.

For example, edge 512 previously connecting 510 and 520 and edge 508previously connecting 510 and 502 can be removed. A new, composite edge1002, connecting map node 520 to map node 502 can be added to maintainconnectivity of the electronic map 1000′. In some embodiments, the noderemoval threshold is greater than the dissimilarity threshold. In someembodiments, the node removal threshold is approximately 50% to 70%dissimilarity.

In some embodiments, updating a map node can involve updating one ormore neighboring map nodes. Referring back to FIGS. 5, 9B and 10B, mapnode 530 can be associated with the position 902 a of self-drivingmaterial-transport vehicle 110. The presence of obstacle 420 b in layout400 can be captured in stored image data of map node 530 in electronicmap 500. When map node 520 is updated based on collected image data atposition 402 showing that the obstacle 420 b is removed, map node 530can also be updated to capture the absence of obstacle 420 b. An updateto the stored image data of map node 530 is indicated by cross-hatchingof map node 530 in electronic map 1000′ of FIG. 10B.

When the electronic map does not include a map node associated with thecurrent positon, then updating the electronic map based at least on thecollected image data involves adding a new map node to the electronicmap for the current position and storing the collected image data at thenew map node. In some embodiments, adding a new map node involvesconnecting the new map node to at least one existing map nodes of theelectronic map. In some embodiments, adding a new map node involvesconnecting the new map node to at least two existing map nodes of theelectronic map.

Referring now to FIG. 11A, there is shown an illustration of anelectronic map 1100, according to at least one embodiment. A new mapnode 1110 has been added to the electronic map 1100. Edges 1112 and 1114are also added to the electronic map 1100 in order to connect map node1110 to existing map nodes 1120 and 1130. The collected image data isstored in association with map node 1110, along with the position of theself-driving material-transport vehicle 110 at the time that the imagedata is collected.

In some embodiments, an electronic map can require map nodes to havepositions that are sufficiently distinct from other map nodes. Afteradding a new map node, the method 400 can further involve determiningwhether the electronic map includes one or more excessive neighboringmap nodes relative to the new map node. An excessive neighboring mapnode can be a map node associated with a position that is less than anexcessive neighbor distance from the current position of the new mapnode. In response to determining that the electronic map includes one ormore excessive neighboring map nodes, the one or more excessiveneighboring map nodes can be removed from the electronic map. In someembodiments, an excessive neighbor distance can be a minimum distancebetween two map nodes of the electronic map.

Referring now to FIG. 11B, there is shown an illustration of anelectronic map 1100′, according to at least one embodiment. After addingnew map node 1110 in electronic map 1100, the method determines thedistance between new map node 1110 and neighbor map nodes 1120, 1130,and 1140. In this example, the distance between map nodes 1110 and 1140is less than the excessive neighbor distance. Map node 1140 is removed(shown as stippled lines) from the electronic map 1100′ and a compositeedge is formed to connect map nodes 1120 and 1140 and maintain theconnectivity of the graph.

Once the electronic map has been updated, the electronic map can beused. The electronic map can be used for display on a user interface.The electronic map can be used by the self-driving material-transportvehicle 110 to identify and navigate its position in the facility.

Referring to FIG. 12 , there is shown a method 1200 for zone-basedremapping of a facility according to some embodiments. The method 1200may include any or all of the steps described in respect of the method600.

The method 1200 begins at step 1210 when a mission is generated, forexample, using a fleet-management system. For the purposes of describingthe method 1200, a distinction can be made between a “standard” missionand a “remapping” mission, though both types of missions general includeproviding a self-driving material-transport vehicle with instructions toexecute a particular task with respect to particular locations.

According to some embodiments, the fleet-management system may generatea standard mission to be executed by a self-driving vehicle. As usedherein, a “standard” mission is a mission that is generatedindependently of a need to map the facility. In some cases, a typicalexample of a standard mission is a material transport mission, such aspicking up an item at one location and delivering it to anotherlocation.

In the event that a standard mission is generated, then the method 1200may generally be used along with the standard mission in order toprovide for continuous mapping/remapping of the facility.

According to some embodiments, the fleet-management system may generatea remapping mission to be executed by a self-driving vehicle. As usedherein, a “remapping” mission is a mission that is generatedspecifically in order to map or remap part or all of the facility. Forexample, a remapping mission may comprise sending a vehicle to aparticular location or series of locations (e.g. within a defined zone)while executing the continuous mapping/remapping systems and methodsdescribed herein.

The fleet-management system may generate a remapping mission accordingto various criteria. According to some embodiments, a remapping missionmay pertain to a specific location or region within the facility (a“remapping zone”), or to the entire facility itself (e.g. a remappingzone may comprise the entire facility).

According to some embodiments, a criterion for a remapping mission maybe the age of the map data (e.g. a maximum data age threshold). In otherwords, if a certain amount of time has passed since continuousmapping/remapping was last performed, a remapping mission can begenerated. According to some embodiments, this time-based criterion canalso be combined with a location-based criterion such that a maximumdata age threshold can be set for a particular remapping zone within thefacility. For example, a remapping zone may be defined for an areawithin the facility in which large volumes of inventory are regularlybeing moved, and the corresponding maximum data age threshold may be setlow relative to other areas of the facility. In other words, for areasof a facility that are known to incur map changes, remapping missionscan be established in order to ensure that the map is constantlyup-to-date for these areas.

According to some embodiments, a criterion for a remapping mission maybe an event that occurs via an enterprise resource planning (“ERP”)system, a manufacturing execution system (“MES”), or another system(collectively referred to as business-level logic systems), and/or afleet-management system. For example, if a known inventory shipmentcauses a large volume of inventory to be displaced within the facility,the fleet-management system can receive a signal from the business-levellogic system indicated that the event has occurred, and this signal canbe used to trigger the generation of a remapping mission, which maypertain to a remapping zone corresponding to the area(s) in which theinventory was displaced.

In some cases, a hybrid mission may be generated that incorporates aremapping mission into a standard mission. A hybrid mission may begenerated, for example, when a standard mission is required and theexecution of the standard mission is expected to take a vehicle in ornear a remapping zone. In this case, the standard mission can be madehybrid with a remapping mission such that the performs continuousmapping/remapping in the remapping zone while executing towards theobjectives of the otherwise standard mission. According to someembodiments, the vehicle may travel a path that includes the locationspertaining to the “standard mission” aspects of the hybrid mission aswell as locations pertaining to the “remapping mission” aspects of thehybrid mission.

At step 1212, the self-driving vehicle receives the mission from thefleet-management system. Generally, the mission comprises one or morelocations associated with the map. The mission may be any of a standardmission, remapping mission, or hybrid mission.

At step 1214, the vehicle autonomously drives according to the mission.According to some embodiments, this comprises autonomously planning apath based on the vehicle's location, the mission (e.g. missionlocations), and the electronic map.

At step 1216, the vehicle determines that it is located within aremapping zone. According to some embodiments, the vehicle may always(i.e. at every location) determine that it is in a remapping zone bydefault. For example, if the vehicle is executing a standard mission andis continuously mapping/remapping while executing the standard mission,the step 1216 is effectively moot, and the method continuous to step1218.

According to some embodiments, the vehicle may only perform continuousmapping/remapping within particular remapping zones within the facility,which may be correlated with the electronic map. For example, if thevehicle is executing a standard mission, the vehicle may only executecontinuous mapping/remapping while it is executing the standard missionand is travelling in a remapping zone while executing the standardmission.

According to some embodiments, the vehicle may only perform continuousmapping/remapping within particular remapping zones within the facilitythat are defined in a remapping mission. For example, if the vehicle isexecuting a remapping mission, the vehicle may only execute continuousmapping/remapping while it is executing the remapping mission and istravelling in the remapping zone associated with (or identified in) theremapping mission.

According to some embodiments, the vehicle may only perform continuousmapping/remapping within remapping zones within the facility that arecomprise or are within a particular proximity to mission locations. Forexample, if the vehicle is executing a hybrid mission, the vehicle mayonly execute continuous mapping/remapping while it is within a remappingzone that it has encountered while traveling to or from standard missionlocations. In other words, while travelling to or from standard missionlocations, the vehicle may intentionally take a detour through a nearbyremapping zone and perform continuous mapping/remapping within thatremapping zone.

At step 1218, the vehicle updates the electronic map of the facilitywithin the remapping zone as generally described herein. According tosome embodiments, step 1218 may comprise the method 600 as previouslydescribed, or any set of steps or variations of the method 600.

It will be appreciated that numerous specific details are set forth inorder to provide a thorough understanding of the example embodimentsdescribed herein. However, it will be understood by those of ordinaryskill in the art that the embodiments described herein may be practicedwithout these specific details. In other instances, well-known methods,procedures and components have not been described in detail so as not toobscure the embodiments described herein. Furthermore, this descriptionand the drawings are not to be considered as limiting the scope of theembodiments described herein in any way, but rather as merely describingthe implementation of the various embodiments described herein.

It should be noted that terms of degree such as “substantially”, “about”and “approximately” when used herein mean a reasonable amount ofdeviation of the modified term such that the end result is notsignificantly changed. These terms of degree should be construed asincluding a deviation of the modified term if this deviation would notnegate the meaning of the term it modifies.

In addition, as used herein, the wording “and/or” is intended torepresent an inclusive-or. That is, “X and/or Y” is intended to mean Xor Y or both, for example. As a further example, “X, Y, and/or Z” isintended to mean X or Y or Z or any combination thereof.

It should be noted that the term “coupled” used herein indicates thattwo elements can be directly coupled to one another or coupled to oneanother through one or more intermediate elements.

The embodiments of the systems and methods described herein may beimplemented in hardware or software, or a combination of both. Theseembodiments may be implemented in computer programs executing onprogrammable computers, each computer including at least one processor,a data storage system (including volatile memory or non-volatile memoryor other data storage elements or a combination thereof), and at leastone communication interface. For example and without limitation, theprogrammable computers may be a server, network appliance, embeddeddevice, computer expansion module, a personal computer, laptop, awireless device or any other computing device capable of beingconfigured to carry out the methods described herein.

Each program may be implemented in a high level procedural or objectoriented programming and/or scripting language, or both, to communicatewith a computer system. However, the programs may be implemented inassembly or machine language, if desired. In any case, the language maybe a compiled or interpreted language. Each such computer program may bestored on a storage media or a device (e.g. ROM, magnetic disk, opticaldisc) readable by a general or special purpose programmable computer,for configuring and operating the computer when the storage media ordevice is read by the computer to perform the procedures describedherein. Embodiments of the system may also be considered to beimplemented as a non-transitory computer-readable storage medium,configured with a computer program, where the storage medium soconfigured causes a computer to operate in a specific and predefinedmanner to perform the functions described herein.

Furthermore, the system, processes and methods of the describedembodiments are capable of being distributed in a computer programproduct comprising a computer readable medium that bears computer usableinstructions for one or more processors. The medium may be provided invarious forms, including one or more diskettes, compact disks, tapes,chips, wireline transmissions, satellite transmissions, internettransmission or downloadings, magnetic and electronic storage media,digital and analog signals, and the like. The computer useableinstructions may also be in various forms, including compiled andnon-compiled code.

Various embodiments have been described herein by way of example only.Various modification and variations may be made to these exampleembodiments without departing from the spirit and scope of theinvention, which is limited only by the appended claims.

1.-45. (canceled)
 46. A method of operating a self-driving materialtransport vehicle to update an electronic map of a facility while theself-driving material transport vehicle is executing a mission, themethod comprising: receiving, at the self-driving material transportvehicle, one or more mission instructions to initiate the self-drivingmaterial transport vehicle to execute one or more mission tasks at oneor more destination locations within the facility; while theself-driving material transport vehicle navigates within the facility toexecute the one or more mission tasks, concurrently operating theself-driving material transport vehicle to execute an electronic mapupdate task separate from the one or more mission tasks, the electronicmap update task comprising one or more map update instructions tooperate the self-driving material transport vehicle to: collect imagedata at a current position of the self-driving material transportvehicle until the self-driving material transport vehicle arrives at adestination location of the one or more destination locations, the imagedata representing one or more features observable from the currentposition; determine if the electronic map comprises a map nodeassociated with the current position, wherein the electronic mapcomprises a set of map nodes, each map node being associated with adifferent location in the facility, and each map node comprises storedimage data associated with a respective location within the facility, inresponse to determining that the electronic map does not include a mapnode associated with the current position, identify one or moreneighboring map nodes within a neighbor threshold of the currentposition; compare the collected image data with the stored image dataassociated with one or more of (i) the map node associated with thecurrent position, and (ii) at least one neighboring map node of the oneor more neighboring map nodes, to determine a dissimilarity level;determine whether the dissimilarity level exceeds a dissimilaritythreshold; and in response to determining the dissimilarity levelexceeds the dissimilarity threshold, update the electronic map based atleast on the collected image data.
 47. The method of claim 46, whereinthe image data comprises semantic image data.
 48. The method of claim47, wherein the semantic image data comprises human-readable data. 49.The method of claim 46, wherein updating the electronic map based atleast on the collected image data comprises processing the collectedimage data to identify one or more classification types associated withthe collected image data and storing the one or more classificationtypes as semantic image data.
 50. The method of claim 46, whereincomparing the collected image data with the stored image data comprises:determining a number of mismatched points between the collected imagedata and the stored image data; and determining the dissimilarity levelbased on the number of mismatched points and a total number of points.51. The method of claim 46, wherein updating the electronic mapcomprises: replacing the stored image data of the map node with thecollected image data.
 52. The method of claim 46, wherein the electronicmap update task further comprises the one or more map updateinstructions to operate the self-driving material transport vehicle to:determine the electronic map does not include the map node associatedwith the current position, add a new map node to the electronic map forthe current position, and store at the new map node the collected imagedata.
 53. The method of claim 52, wherein the electronic map update taskfurther comprises the one or more map update instructions to operate theself-driving material transport vehicle to: determine the electronic mapincludes one or more excessive neighboring map nodes relative to the newmap node, the one or more excessive neighboring map nodes being a mapnode associated with a position that is less than an excessive neighbordistance from the current position of the new map node; and in responseto determining the electronic map includes the one or more excessiveneighboring map nodes, remove the one or more excessive neighboring mapnodes from the electronic map.
 54. The method of claim 53, wherein theexcessive neighbor distance comprises a minimum distance between two mapnodes of the electronic map.
 55. The method of claim 46, wherein theelectronic map update task further comprises the one or more map updateinstructions to operate the self-driving material transport vehicle to:for each neighboring map node of the one or more neighboring map nodes,compare the collected image data with the stored image data of thatneighboring node of the one or more neighboring map nodes to determinethe dissimilarity level; and in response to determining thedissimilarity level exceeds a node removal threshold, remove thatneighboring map node of the one or more neighboring map nodes from theelectronic map.
 56. The method of claim 46, wherein the neighborthreshold comprises a maximum distance between a position of two mapnodes.
 57. The method of claim 46, wherein the dissimilarity thresholdcomprises a minimum mismatch between the collected image data and thestored image data to indicate the electronic map is outdated.
 58. Themethod of claim 46, wherein the dissimilarity threshold is based on atleast one of a facility type of the facility and one or more priorupdates to the electronic map.
 59. The method of claim 46, furthercomprising determining if the self-driving material transport vehicle isin a remapping zone of the facility and in response to determining theself-driving material transport vehicle is within a remapping zone ofthe facility, operating the self-driving material transport vehicle toexecute the electronic map update task.
 60. The method of claim 46,wherein the dissimilarity level exceeds the dissimilarity threshold ifthe collected image data indicates the addition or removal of at leastone obstacle, and wherein the at least one obstacle is at least onenon-dynamic object that obstructs navigation of the self-drivingmaterial-transport vehicle; and wherein the updating the stored imagedata is based at least on the addition or removal of at least oneobstacle indicated by the collected image data at the current position.61. A system for operating a self-driving material transport vehicle toupdate an electronic map of a facility while the self-driving materialtransport is executing a mission, the system comprising: one or moreself-driving material transport vehicles operable to navigate thefacility, a self-driving material-transport vehicle of the one or moreself-driving material transport vehicles comprising: a memory to storethe electronic map, the electronic map comprising a set of map nodes,each map node being associated with a different location in thefacility, and each map node comprises a stored image data associatedwith a respective location within the facility; one or more sensors tocollect image data at a current position of the at least oneself-driving material transport vehicle until the self-driving materialtransport vehicle arrives at a destination location of one or moredestination locations within the facility, the image data representingone or more features observable from the current position; and aprocessor operable to: receive one or more mission instructions toinitiate the self-driving material transport vehicle to execute one ormore mission tasks at the one or more destination locations within thefacility; while the self-driving material transport vehicle navigateswithin the facility to execute the one or more mission tasks,concurrently execute an electronic map update task separate from the oneor more mission tasks, the electronic map update task comprising one ormore map update instructions to: determine if the electronic mapcomprises a map node associated with the current position; in responseto determining that the electronic map does not include a map nodeassociated with the current position, identify one or more neighboringmap nodes within a neighbor threshold of the current position; comparethe collected image data with the stored image data associated with oneor more (i) the map node associated with the current position, and (ii)at least one neighboring map node of the one or more neighboring mapsnodes, to determine a dissimilarity level; determine whether thedissimilarity level exceeds a dissimilarity threshold; and in responseto determining the dissimilarity level exceeds the dissimilaritythreshold stored in the memory, update the electronic map based at leaston the collected image data.
 62. The system of claim 61, wherein theimage data comprises semantic image data.
 63. The system of claim 62,wherein the semantic image data comprises human-readable data.
 64. Thesystem of claim 61, wherein updating the electronic map based at leaston the collected image data comprises processing the collected imagedata to identify classification types associated with the collectedimage data and storing the one or more classification types as semanticimage data.
 65. The system of claim 61, wherein the electronic mapupdate task further comprises the one or more map update instructionsto: determine a number of mismatched points between the collected imagedata and the stored image data; and determine the dissimilarity levelbased on the number of mismatched points and a total number of points.66. The system of claim 61, wherein the electronic map update taskfurther comprises the one or more map update instructions to replace thestored image data of the map node associated with the current positionwith the collected image data.
 67. The system of claim 61, wherein theelectronic map update task further comprises the one or more map updateinstructions to, determine the electronic map does not include the mapnode associated with the current position, add a new map node to theelectronic map for the current position and store at the new map nodethe collected image data.
 68. The system of claim 61, wherein theelectronic map update task further comprises the one or more map updateinstructions to: determine the electronic map includes one or moreexcessive neighboring map nodes relative to the new map node, the one ormore excessive neighboring map nodes being a map node associated with aposition that is less than an excessive neighbor distance from thecurrent position of the new map node, the excessive neighbor distancedefining a minimum distance between two map nodes of the electronic map;and in response to determining the electronic map includes the one ormore excessive neighboring map nodes, remove the one or more excessiveneighboring map nodes from the electronic map.
 69. The system of claim68, wherein the excessive neighbor distance comprises a minimum distancebetween two map nodes of the electronic map.
 70. The system of claim 61,wherein the electronic map update task further comprises the one or moremap update instructions to, for each neighboring map node of the one ormore neighboring map nodes, compare the collected image data with thestored image data of that neighboring node of the one or moreneighboring map nodes to determine the dissimilarity level; and inresponse to determining the dissimilarity level exceeds a node removalthreshold, remove that neighboring map node of the one or moreneighboring map nodes from the electronic map.
 71. The system of claim61, wherein the neighbor threshold comprises a maximum distance betweena position of two map nodes.
 72. The system of claim 61, wherein thedissimilarity threshold comprises a minimum mismatch between thecollected image data and the stored image data to indicate theelectronic map is outdated.
 73. The system of claim 61, wherein thedissimilarity threshold is based on at least one of a facility type ofthe facility and one or more prior updates to the electronic map. 74.The system of claim 61, wherein the processor is further operable todetermine if the self-driving material transport vehicle is in aremapping zone of the facility and in response to determining theself-driving material transport vehicle is within a remapping zone ofthe facility, execute the electronic map update task.
 75. The system ofclaim 61, wherein the dissimilarity level exceeds the dissimilaritythreshold if the collected image data indicates the addition or removalof at least one obstacle, and wherein the at least one obstacle is atleast one non-dynamic object that obstructs navigation of theself-driving material transport vehicle; and wherein the electronic mapupdate task further comprises the one or more map update instructions toupdate the stored image data based at least on the addition or removalof at least one obstacle indicated by the collected image data at thecurrent position.